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NC  A&T  Algebra  Tutorial  Dialogue  Project: 

ONR  Grant  Final  Report 

The  goals  of  the  NC  A&T  algebra  tutorial  dialogue  project  are  to  collect,  analyze,  and  verify 

protocols  from  the  human  tutoring  of  algebra  in  a  manner  consistent  with  current  work  adapting 

human  tutoring  dialogues  for  intelligent  tutoring  systems. 

1.  Accomplishments  Summary 

With  regards  to  collecting  data,  we  have  accomplished: 

1.  Recruited  and  tutored  62  students  from  January  2002  to  November  2003. 

2.  Accumulated  51  keyboard  tutoring  transcripts,  5  video  taped  face-to-face  sessions,  and  55  sets 
of  pre-  and  post-tests. 

3.  Used  1  expert  tutor  (24  transcripts),  2  experienced  tutors  (16  transcripts),  and  1  novice  tutor 
(11  transcripts). 

With  regards  to  data  analysis,  we  have  accomplished: 

4.  Completed  a  preliminary  mark-up  of  all  problems  in  8  transcripts. 

5.  More  fully  analyzed  2  algebra  problems  across  many  transcripts,  resulting  in  the  identification 
of  six  tutorial  methods  and  a  number  of  associated  topics. 

6.  Accumulated  a  preliminary  collection  of  methods,  topics,  and  language  for  one  problem  for 
the  semi-automated  Wooz  tutor. 

7.  Analyzed  learning  gains,  time  per  problem,  number  of  problems  worked  on,  number  of  turns 
per  problem,  etc.  vs.  student  classroom  achievement  and  tutor  experience  level 
(expert/experienced/novice). 

8.  Analyzed  student  initiatives  in  keyboard  vs.  spoken  CIRCSIM-Tutor  transeripts,  and  started 
applying  this  analysis  to  our  algebra  transcripts. 


With  regard  to  validating  our  markup  we  have  aecomplished: 

9.  Used  the  Wooz  tutor  to  evaluate  our  analysis  of  the  one  problem  by  tutoring  students. 

With  regards  to  software  development,  we  have  accomplished: 

10.  Developed  the  iTutor  computer-mediated  tutoring  application,  with  ancillary  software  for 
researchers  to  navigate  among,  inspect,  and  print  out  human-readable  transcripts. 

11.  Wrote  the  Wooz  tutor,  the  computer-assist  for  human  tutoring  that  validates  our  transcript 
analysis. 


2.  Approach 

1.  Collection  of  Transcripts 

Students  were  recruited  fi'om  the  first-year  algebra  class  at  NC  A&T  State  University.  The 
students  in  this  class  are  largely  non-technical  majors.  The  students  were  categorized  by  their 
teacher  into  one  of  five  levels  of  academic  achievement,  based  on  their  prior  performance  in  the 
algebra  class. 

For  our  tutoring  sessions,  which  were  usually  one  hour  long,  the  mathematics  tutoring  center 
at  NC  A&T  provided  a  venue.  Our  expert  tutor  was  Dr.  Kathy  Cousins-Cooper,  a  Ph.D.  in 
mathematics  education  and  co-PI  on  this  grant,  the  two  experienced  tutors  came  from  the  staff  of 
the  tutoring  center  but  have  no  formal  training  in  mathematics  education,  and  the  novice  tutor 
was  an  upper-level  mathematics  student. 

Typical  problems  solved  during  these  tutoring  sessions  include  both  symbolic  manipulation 
problems  and  word  problems,  viz: 

1 .  Please  factor  3x^  - 1 1  x  +  6. 
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2.  Bob  drove  “m”  miles  from  Denver  to  Fargo.  Normally  this  trip  takes  “n”  hours,  but  on 
Tuesday  there  was  good  weather  and  he  saved  2  hours.  Write  an  equation  for  his  driving 
speed  “s”. 

With  every  tutoring  session  we  administered  pre-  and  post-tests.  The  pre-tests  were  used  for 
determining  the  highest  priority  topics  for  tutoring.  The  tutor  picked  those  topics  where  the 
student  gave  the  wrong  answer,  but  did  not  leave  the  test  sheet  completely  blank.  We  felt  that 
these  were  the  areas  where  the  student  was  most  likely  ready  to  benefit  from  instruction.  The 
post-test  problems  presented  to  a  student  were  restricted  to  the  problem  areas  that  were  tutored, 
so  the  learning  gains  recorded  represent  gains  for  the  tutored  topics  only. 

The  keyboard-to-keyboard  transcripts  were  collected  using  the  iTutor  computer-mediated 
tutoring  program,  constructed  by  this  project.  This  software  was  later  upgraded  to  become  the 
Wooz  tutor.  Our  transcripts  were  collected  with  the  tutor  and  student  in  separate  rooms  so  there 
could  be  no  interaction  during  the  session  beyond  what  was  captured  by  the  computer. 

We  regard  it  as  a  high  priority  to  put  our  transcripts  in  a  usable,  readable  form,  and  make  them 
available  for  dissemination  to  other  researchers.  To  this  end  we  developed  software  to  make  the 
transcripts  readable,  for  example  converting  the  MathML  machine-readable  representations  of 
equations  into  normal  printed  representations,  and  interpolating  the  equations  into  the  printed 
transcript  of  the  dialogue  stream  at  the  points  where  the  equations  were  updated  We  also  have 
software  for  navigating  among  transcript  turns  on-line. 

In  addition  to  the  keyboard  sessions,  we  have  video  tapes  of  five  tutoring  sessions  with  the 
expert  and  experienced  tutors.  These  sessions  have  not  been  transcribed. 

2.  Finding  Human  Tutoring  Tactics  and  Language 

We  have  been  performing  intention-based  markup  of  human  tutorial  dialogue,  specifically  with 
an  aim  to  discover  schemata  that  describe  the  tutor’s  behavior.  We  are  also  interested  in  the 
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words  and  phrases  the  tutor  uses,  the  observable  trace  of  social  interactions  (e.g.,  the  language 
the  tutor  uses  to  acknowledge  various  student  responses),  the  topics  the  tutor  chooses  to  teach 
and  ignore,  and  other  quantifiable  linguistic  phenomena  such  as  use  of  discourse  markers  and 
styles  of  acknowledgments.  Comparing  transcripts  prepared  by  expert  and  inexpert  tutors  enables 
us  to  focus  on  the  kinds  of  behaviors  that  make  expert  tutors  better. 

Our  transcript  analysis  is  in  the  same  style  as  used  by  the  CIRCSIM-Tutor  project. 

3.  Categorizing  Student  Behaviors 

In  addition  to  categorizing  the  student  answers  to  questions  that  drive  the  normal  course  of 
tutoring,  we  have  been  studying  student  conversational  initiatives  and  the  responses  to  open 
questions.  Handling  student  initiatives  and  responses  to  open  questions  are  an  important  goal  at 
the  edge  of  current  research  in  dialogue-based  tutoring. 

4.  Assessment  of  the  Results 

We  have  built  a  way  to  evaluate  some  of  the  tutoring  strategies,  a  dummy  machine  tutor  (the 
Wooz  tutor)  that  constrains  a  human  tutor  to  converse  using  the  strategies  and  language  we  have 
coded  fi'om  the  transcripts.  We  have  used  two  measures  for  evaluating  the  Wooz  tutor.  The  first 
is  the  pre-  and  post-test  results,  which  we  can  compare  with  the  results  fi’om  our  unaided  human 
tutors.  The  second  comes  fi'om  the  fact  that  the  human  tutor  vets  and  edits  all  language  that  the 
Wooz  tutor  emits,  even  if  it  was  mechanically  constructed.  The  extent  to  which  the  human  tutor 
edits  this  language,  or  abandons  the  mechanically  generated  responses  entirely,  tells  us  how 
successful  die  Wooz  tutor  is  being.  Later,  we  can  use  survey  instruments  to  find  how  the  students 
and  tutors  react  to  Wooz  tutoring  sessions. 

Since  the  Wooz  tutor  is  an  enhanced  version  of  the  iTutor  software,  tutoring  sessions  can  use 
computer-aided  tutoring  for  some  problems  and  the  regular  tutoring  for  other  problems  all  in  the 
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same  tutoring  session.  Thus  we  did  not  need  to  have  markup  and  analysis  of  all  problem  types 
before  we  began  Wooz  tutoring. 

3.  Results 

1 .  T  utoring  Statistics 

From  pre-  and  post-tests  we  have  measured  learning  gains  during  our  tutoring  sessions.  These 
gains  are  calculated  by  (posttest  -  pretest)  /  (100  -  pretest),  where  we  selected  from  the  pre-  and 
post-tests  only  those  topics  that  were  tutored.  Ovct  all  keyboard  sessions,  the  learning  gain 
averages  0.35.  In  the  five  face-to-face  sessions,  the  learning  gain  averaged  0.19,  less  than  in  the 
keyboard-mediated  sessions  but  there  is  not  enough  data  to  say  this  is  significant.  We  conclude 
from  these  results  that  the  tutoring  we  are  studying  is,  in  fact,  having  a  strongly  positive  effect. 
This  validates  that  the  phenomena  we  extract  from  the  transcripts  should  indeed  be  potentially 
usefiil.  Furthermore,  the  expert  tutor  really  is  having  more  effect  than  the  less  experienced  tutors, 
which  means  that  the  expert  vs.  novice  analysis  paradigm  will  focus  on  differences  that  are 
pedagogically  significant. 

Our  students  are  divided  into  five  ability  groupings  according  to  their  classroom  record  prior 
to  the  tutoring.  We  find  that  the  lower  levels  exhibit  somewhat  lower  learning  gains.  The  lower 
achieving  groups  need  more  time  and  turns  to  solve  a  problem  compared  to  higher  achieving 
groups,  and  solve  fewer  problems  during  one  hour  of  tutoring.  These  results  are  summarized  in 
Tablel,  Table2,  and  Table  3. 

Learning  gains  were  also  correlated  with  tutor  experience  level.  The  expert  tutor  showed  an 
average  gain  of  0.41  over  24  sessions,  the  experienced  tutors’  learning  gain  is  0.35  over  their  16 
sessions,  and  the  novice  tutor's  learning  gain  is  0.1 8  over  their  1 1  sessions. 
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Overall,  students  solved  on  the  average  of  5.1  problems  per  keyboard  session,  and  11  in  the 
face-to-face  sessions. 


Table  2 


level 


Average  of  needed  time  &  #  of  turns  per  problem  vs.  student  level 


avg,  need®*  time  (m  probfem 

of  turns  per  prahtem  ; 

18:23.8 

19.93 

15:27.6 

18.53 

09:38.6 

14.43 

13:03.1 

11.93 

09:11.6 

10.44 

14:09.3 

16.15 

2.  Transcript  Analysis 

Tutorial  Goals  and  Language.  From  the  transcripts  we  have  learned  quite  a  bit  about  the 
structure  of  our  algebra  tutoring.  The  most  common  structure  is  to  1)  make  sure  the  student  is 
aware  of  what  operation  is  being  asked,  2)  make  sure  the  student  is  aware  of  the  approach  to 
take,  and  3)  make  sure  the  problem  is  solved. 


Figure  1  shows  an  extract  from  a  relatively  short  dialogue  where  the  student  solved  one 
multiplication  problem.  (In  printed  transcripts,  the  evolving  equation  in  the  equation  window  is 
interpolated  into  the  dialogue  every  time  the  equation  changes.)  It  well  illustrates  the  strategy  of 
ensuring  that  the  student  can  state  the  type  of  problem  (multiplying  polynomials  in  this  case)  and 
a  technique  to  solve  it  (a  mnemonic  device  in  this  case).  This  tactic  alone  is  often  enough  to  get 
the  student  to  solve  the  problem,  as  illustrated,  even  thou^  the  student  failed  to  solve  similar 
problems  on  the  pre-test.  Getting  the  student  to  explicitly  state  the  problem  and  method  is 
consistent  with  the  view  that  learning  mathematics  often  invokes  metacognitive  processes  for 
just  these  sorts  of  operations. 


Problem:  Solve  (3x-2Yx+5) 

Tl-tu-101-1:  What  operation  do  you  need  to  apply  here? 

Tl-st-102-1:  Factoring. 

Tl-tu-103-1 :  No,  Not  factoring. 

Tl-tu-103-2:  The  expression  is  already  factored. 

Tl-tu-103-3:  What  is  the  opposite  of  factoring? 

T  1-st- 104-1:  Distributing. 

Tl-tu-105-1:  You  can  say  distributing. 

Tl-tu-105-2:  But  we  are  multiplying  here. 

Tl-tu-105-3:  Now,  what  process  do  we  use  when  we  are  multiplying  two  binomials. 
Tl-st-1 06-1:  Foil. 

T 1 -tu- 1 07- 1 :  Excellent,  You  use  FOIL. 

Tl-tu-107-2:  What  does  FOIL  stand  for? 

Tl-st-108-1 :  First  Outer  Inner  Last. 

T1  -tu-1 09-1 :  Good,  Now  go  ahead  and  use  FOIL  to  multiply  the  two  binomials  together. 
Tl-st-1 10-1:  3x^+15x-2x-10 
Tl-st-1 11-1:  3x^+13x-10 
T1 -tu-1 13-1:  Excellent. 

Figure  1 :  Excerpt  of  tutoring  dialogue  for  multiplication  problem 


Each  goal  has  subgoals,  usually  involving  eliciting  some  responses  from  the  student,  that  we 
call  tactics.  Some  of  the  tutoring  tactics  we  have  identified  and  annotated  are:  “tutoring  via 
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FOIL”  (a  mnemonic  for  multiplying  binomials),  “correcting  via  factors”  (having  the  student 
factor  individual  coefficients  of  a  polynomial),  “moving  forward”  (letting  the  student  proceed 
to  see  what  happens),  “making  binomials”  (mechanically  setting  up  the  form  of  the  result,  for 
the  student  to  fill  in),  and  “trial  and  error”  (having  the  student  guess  a  result  and  verify  whether 
it  is  correct).  Figure  2  shows  the  markup  of  a  more  substantial  tutoring  episode,  illustrating  these 
tactics. 


Problem:  factor  2x^  -7x  +6 

<T -identifies-operation> 

<T-tutors-operation> 

<T-elicits  DM=now> 

T:  Now  what  are  we  doing  here? 

<S-ans  catg=incorrect> 

S:  Solving  the  quadratic  equation.  </S-ans> 
</T-elicits> 

</T -tutors-operation> 

<T -tutors-expression> 

<T-elicits> 

T:  Is  this  an  equation? 

<S-ans  catg=correct> 

S:  no,  because  there  isn  *t  an  equal  sign.  </S-ans> 
<T-ack  type=positive> 

T:  Thatis  right  </T-ack> 

</T-elicits> 

<T-informs> 

T:  This  is  an  expression. 

</T-informs> 

</T-tutors-expression> 

<T-tutors-operation  attempt=2> 

<T-elicits  DM=so> 

T:  So  what  are  we  doing  with  this  egression? 
<S-ans  catg=correct> 

S:  factoring.  </S-ans> 

<T-ack  type=positive> 

T:  Good  </T-ack> 

</T-elicits> 

</T-tutors-operation> 

</T-identifies-operation> _ _ _ 

Figure  2:  Excerpt  of  annotated  transcript 
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The  goal  structure  of  this  dialogue  can  be  summarized  as  follows: 


Goal  Hierarchy  Comment 


Identify  operation  Operation  is  factoring 
T  UTOR  OPERATION  Elicit  the  operation  from  the  student,  1st  attempt 

T  UTOR  EXPRESSION  Student  did  not  recognize  it  is  an  expression,  this  is  a  fixup 

Tutor  OPERATION  2nd  attempt 

Identify  approach  Solution  approach  is  trial-and-error  factoring 

Tutor  trial-and-error  Elicit  from  or  inform  the  student  to  factor  by  trial-and-error 
Solve  problem 

Make  binomials  Ask  the  student  to  produce  both  binomial  terms  all  at  once. 

Obtain  factors  Student  failed,  obtain  terms  one  at  a  time,  1st  attempt 

Obtain  first  factor 
Obtain  second  factor 
Confirm  factoring 

ChECK-VIA-FOIL  Confirm  by  multiplying  back  The  student  made  an  error. 

Obtain  factors  2nd  attempt 

Obtain  second  factor  Correct  the  one  term  the  student  got  wrong 

(Confirm  factoring)  Student  did  this  without  the  tutor  prompting _ 


Following  the  identification  of  schemata,  we  collected  examples  of  language  used  for  each 
goal.  The  set  of  goals  and  associated  sentences  are  then  collected  together,  one  set  for  each 
problem.  Some  of  the  sentences  are  simple  templates  where  the  variable  slots  can  be  filled  in 
with  the  student's  name  or  problem-specific  information.  An  extract  from  the  set  of  goals  for  the 
factoring  problem,  with  a  few  of  the  sentences  identified  for  each  goal,  is  shown  in  Figure  3. 
This  data  is  what  the  human  tutor  picks  fi-om  when  tutoring  using  the  Wooz  tutor. 
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Identify  THE  PROBLEM 

What  are  we  doing  with  this  expression? 

We  are  factoring. 

Introduce  the  trail-error  approach. 

What  method  can  we  use  to  factor  this? 

Let’s  use  the  trial  and  error  method. 

Solve  problem 
Solve  via  making  binomials 
Make  binomials 

So  you  set  up  your  2  sets  of  parentheses  and  fill  them  in  with  factors  of  the  first 

term  and  the  last. 

This  trinomial  will  factor  into  2  binomials 
Confirm  factoring 

Now  how  do  we  check  our  factors. 

Now  use  FOIL  to  check  this 
Make  FIRST  TERM 

What  are  the  two  factors  of  2x  squared? 

Ok,  X  squared  factors  into  x  and  x,  right? 

Solve  via  correcting  factors 
Corrects  TERM 
Corrects  2nd  term 
Corrects  factor  sign 

Solve  via  move  forward 

_ Go  ahead  and  solve  it _ 

Figure  3:  Excerpt  from  goals  and  associated  sentences  for  factoring  problem 


Student  Initiatives.  Our  analysis  of  student  initiatives  started  with  a  comparison  of  student 
behavior  between  keyboard  and  in-person  tutoring  in  CIRCSIM-Tutor  transcripts.  Our  data 
shows  that  while  the  initiatives  are  similar  in  some  categories,  the  differences  in  the 
communication  medium  and  conversational  setting  result  in  considerable  differences  both  in 
kinds  and  numbers  of  student  initiatives.  In  the  face-to-face  verbal  transcripts,  there  were  more 
student  requests  for  information  and  more  tutor  interruptions  of  the  student.  Our  preliminary 
analysis  of  the  algebra  keyboard-to-keyboard  transcripts  shows  that  they  are  similar  to  the 
CIRCSIM-Tutor  keyboard-to-keyboard  ones  in  these  regards. 
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3.  Evaluating  Transcript  Markup  with  the  Wooz  Tutor 

The  binomial  factoring  problem  was  the  only  problem  sufficiently  analyzed  that  we  could 

evaluate  our  analysis  with  the  Wooz  tutor.  The  machine  tutor  provided  assistance  to  the  hxmian 
tutor  on  this  problem  during  six  tutoring  sessions,  compared  with  fourteen  sessions  where  the 
problem  was  tutored  without  machine  assistance. 

In  Wooz  tutor  operation,  the  human  tutor  observes  a  tree-structured  map  of  dialogue  goals. 
Clicking  on  a  goal  reveals  a  menu  of  sentences  that  the  tutor  can  pick  fi'om.  These  sentences  are 
then  subject  to  editing  before  the  tutor  submits  them  into  the  dialogue.  From  the  student’s 
viewpoint,  there  is  no  difference  between  the  computer-assisted  dialogue  and  the  regular 
dialogue. 

The  average  learning  gain  on  the  one  Wooz-tutored  problem  type  was  0.75  for  the  computer- 
assisted  tutoring  (six  cases)  and  -0.16  (a  loss)  for  the  sessions  where  the  same  problem  received 
unassisted  tutoring  (15  cases).  We  noticed  that  the  six  Wooz-tutor  sessions  were  all  at  the  lower 
3  of  our  5  academic  achievement  levels,  which  affects  expected  learning  gain.  Matching  on 
student  achievement  level,  the  unassisted  tutoring  of  this  same  problem  (10  cases)  exhibits  a 
learning  gain  of  0.0. 


Table  4.  Wooz-tutoring  vs.  imassisted  human  tutoring 


Wooz-Assisted 

Dialogues 

All  Manual 
Dialogues 

Achievement-Matched- 

Dialogues 

Learning  Gain 

0.75 

-0.16 

0.00 

Avg.  Turns 

21.83 

20.13 

25.30 

Avg.  Time 

18:52 

14:05 

18:04 

N 

6 

15 

10 
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It  is  not  clear  to  us  why  the  Wooz  computer-assisted  tutoring  was  so  much  better  than 
unassisted  tutoring.  There  is  not  enough  data  to  get  statistical  significance,  so  we  can  not  be  sure 
the  increased  learning  gains  are  a  reproducible  effect.  We  speculate,  however,  that  in  the  process 
of  examining  the  transcripts,  particularly  with  the  focus  on  this  one  problem,  the  tutors  improved 
their  performance  on  the  problem  type  in  question.  This  effect  would  also  explain  why  the  face- 
to-face  transcripts  exhibited  less  of  a  learning  gain  than  the  keyboard-to-keyboard  transcripts,  as 
they  were  all  collected  and  examined  at  the  start  of  the  project. 

The  other  way  to  evaluate  the  markup  was  to  observe  how  frequently  the  human  tutor  edited  or 
replaced  the  Wooz  tutor-suggested  language  before  submitting  it  to  the  student.  In  fact,  the  tutors 
almost  always  followed  the  suggested  tutorial  goal  schemata.  This  suggests  that  we  have  the  goal 
structure  correct.  We  have  not  tried  the  computer-suggested  goal  structure  and  dialogue  with 
novice  tutors  to  see  whether  it  affects  their  tutoring. 

Of  the  tutor  turns  in  the  Wooz-assisted  dialogue,  70%  were  extracted  from  the  database  of 
canned  sentences  with  no  change,  6%  were  edits  of  existing  sentences,  and  24%  were  new 
sentences.  There  seems  little  observable  qualitative  difference  between  the  edits  and  the  new 
sentences,  it  seems  that  once  the  tutor  started  editing  a  sentence  she  changed  almost  the  whole 
thing.  The  new  and  changed  sentences  were  still  in  service  of  the  same  tutorial  methods. 
However  the  new  and  changed  sentences  almost  always  respond  to  specifics.,  of  student 
utterances  tiiat  did  not  appear  in  the  attested  transcripts  used  in  building  the  sentence  database. 
Here  is  an  example  of  a  modified  turn: 

Stu:  I'm  going  to  use  the  quadratic  formula. 

Tut  (original):  Is  this  an  equation? 

Tut  (edited);  We  use  the  quadratic  formida  for  quadratic  equations.  Is  this  an  equation ? 
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This  phenomenon,  the  hximan  tutor  responding  to  specific  variations  in  the  student  responses, 
would  seem  to  reduce  the  Wooz  tutor's  evaluative  probity.  When  a  tutor  changes  a  sentence,  we 
have  no  way  to  know  whether  the  unchanged  sentence  would  have  worked  just  as  well. 
Neverfiieless,  with  experience  we  should  build  up  knowledge  of  what  rates  of  sentaice 
modifications  to  reasonably  expect. 

4.  Software  Application  Development 

One  result  of  this  project  is  that  we  have  developed  software  that  will  be  of  use  to  us  in  the  future 
and  possibly  to  other  similar  projects,  the  iTutor/Wooz  collection  of  programs  for  computer- 
mediated  keyboard  to  keyboard  tutoring  of  algebra.  These  programs  support  a  model  of  tutoring 
where  the  tutor  and  student  collaboratively  solve  algebra  problems.  The  equations  being  worked 
on,  the  dialogue,  and  the  original  problem  statement,  are  visible  in  separate  windows.  Both  the 
tutor  and  student  can  edit  the  equation  and  contribute  the  dialogue. 

In  more  detail,  the  tutor  and  student  use  almost  identical  screens,  illustrated  in  Figure  4.  The 
“load  question”  button  appears  on  the  tutor’s  side  only,  enabling  the  tutor  to  pick  fi-om  a 
collection  of  pre-canned  problems  which  are  loaded  into  the  problem  statement  and  equation 
areas.  Each  question  customizes  the  equation-editing  toolbar,  deactivating  mathematical 
operations  that  are  not  useful  for  this  problem.  The  dialogue  is  interleaved  in  a  chat  window. 
During  the  dialogue,  the  two  parties  cannot  type  simultaneously,  so  the  application  must  keep 
track  of  who  is  taking  a  turn.  The  tutor  can  seize  the  tum-taking  state,  interrupting  the  student, 
but  not  vice  versa.  After  a  turn,  the  software  returns  to  a  quiescent  state,  so  either  party  can 
continue  the  conversation. 
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J^MwMsfion 


i|  SoMng  Linear  Ec^aions 
4  Solve  «te  e<?ttatoft :  reSsf ewes  formrta  (E) 


sfion  Release  turn  .j  SiicwCelDiiiv  I 


ProWem/Question  Area 


ipitoi:  HI  James.  Oe  lewwrHSwte  start  ws 

ij^Ktent  ust  ita^.  Can  w  glvt  me  mm  Iniai  s^s"? 


ChatAiea 


^^eartiwi^lmhasiNrtrtm  ; 

Figure  4:  iXutor  client  interface 


The  Wooz  tutor  upgrade  to  the  basic  iTutor  application  works  as  follows.  From  the  student’s 
screen,  it  looks  tiie  same  as  iTutor  for  computer-mediated  human  tutoring.  However  on  the 
tutor’s  screen  there  is  a  tree  structure  of  tutoring  goals.  Expanding  a  node  on  the  tree  yields  a 
selection  of  sentences  the  tutor  can  pick.  Some  of  the  sentences  are  automatically  filled  in  with 
information  from  the  current  tutoring  environment,  for  example  parts  of  the  problem  equation  or 
the  name  of  the  student.  (Some  of  our  tutors  occasionally  address  the  students  by  name.  Since 
this  is  possibly  a  significant  motivational  technique,  we  do  not  want  to  exclude  that  behavior 
from  our  packaged  tutoring  protocols.)  Other  parts  of  the  chosen  sentence  can  be  filled  in  as 
needed,  for  example  discourse  cue  words  and  acknowledgments  to  the  student.  The  whole 
sentence  can  be  edited  or  replaced  before  the  tutor  sends  it  to  the  student.  The  pre-packaged 
sentence  is  almost  always  picked  from  attested  sentences  in  the  transcripts.  The  tutors’  job  is  to 
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navigate  through  the  tree  hierarchy,  picking  alternatives  or  retrying,  or  even  abandoning  the 
structure,  as  they  see  fit. 

The  iTutorAVooz  software  application  is  developed  in  JAVA™  using  a  client-server 
architecture.  The  same  client  program  runs  on  two  different  computers  in  two  different  rooms, 
one  for  the  tutor  and  one  for  the  student.  The  server  program  runs  on  potentially  yet  a  third 
computer,  managing  communication  between  the  tutor  and  student  screens  and  logging  all 
interactions  in  disk  files.  All  communication  is  via  TCP/IP,  and  the  user  interface  screens  are 
written  with  standard  Java  interface  classes,  so  the  software  is  largely  platform-independent. 

Among  the  ancillary  software  is  a  transcript  reader  program,  and  software  for  turn-numbering 
and  printing  in  readable  form.  The  log  reader  allows  the  user  to  navigate  and  inspect 
conversations.  Among  its  features  is  an  ability  to  display  the  differences  between  successive 
versions  of  an  equation,  so  the  user  can  observe  what  in  particular  the  student  is  changing. 

For  tile  Wooz  tutor,  we  also  have  ancillary  software  for  gathering  sentences  and  goal  structures 
into  a  database,  which  is  accessed  during  tutorial  operation. 

4.  Dissemination  and  Continuation 

Transcripts  of  our  tutorial  dialogues  will  be  submitted  to  repositories  of  such  transcripts,  for 
example  the  collection  run  by  the  Circle  project  at  LRDC  which  already  contains  Neal 
Heffeman’s  algebra  tutorial  dialogues. 

We  have  submitted  a  proposal  to  the  Education  Department’s  MSEIP  program  to  use  our 
project’s  facilities  and  expertise  for  improving  the  education  of  science  and  engineering  students. 
The  idea  is  to  train  science  and  engineering  students  to  tutor  elementary  algebra,  thereby 
improving  their  own  skills  in  commvmication  and  training  and  exercising  their  metacognitive 
abilities  in  mathematics.  To  this  end,  they  would  use  our  tutoring  setup:  the  iTutor/Wooz 
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interface,  the  pre-  and  post-tests,  the  mathematics  tutoring  center,  and  project  personnel.  The 
students  would  then  be  able  to  1)  review  in  detail  (with  experts)  transcripts  of  their  own  tutoring, 
3)  measure  their  performance,  and  3)  inspect  the  structure  of  known-good  tutoring  interactions. 

We  have  submitted  several  proposals,  with  no  success  to  date,  to  continue  and  expand  the 
work  described  here. 

5.  Statistics 

Number  of  Degrees  Granted :  Two  Master  of  Science  degrees  (Niraj  Patel,  lie  Zhao) 

Number  of  Undergraduate  Students  employed:  Four 

PI/CoPI  Minority  Women:  One  as  a  Co-PI  (Dr.  Kathy  Cousins-Cooper) 

PI/CoPI  Non-Minority  Women:  One  as  a  PI  (Dr.  Jung  Hee  Kim) 
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